Image processing device, control system, control method of image processing device, control program, and recording medium

ABSTRACT

An image processing device includes a predicted position calculator configured to predict a current position of a target object, an obtained image analyzer configured to search for a position of the target object from a predetermined region that is a partial region of a captured image and that includes a position predicted by the predicted position calculator, and an analysis result outputting unit configured to output the position of the target object retrieved by the obtained image analyzer.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on Japanese Patent Application No. 2017-005993filed with the Japan Patent Office on Jan. 17, 2017, the entire contentsof which are incorporated herein by reference.

FIELD

The disclosure relates to an image processing device for positioncontrol, a control method of the image processing device, a controlsystem for position control, a control program, a recording medium andthe like.

BACKGROUND

An image processing technique is known to be used by an inspectiondevice or a measurement device in a manufacturing site. For example, adevice that performs positioning by using an image processing techniqueso as to process or inspect a target object is known.

Patent Document 1 describes a manufacturing device that is configured toperform positioning by controlling a conveying section based on theposition of a positioning mark.

Also, Patent Document 2 describes a control system that is configured toposition a target object at a target position by using image data thatis acquired by capturing an image of the target object provided with acharacteristic portion for positioning.

[Patent Document 1] JP 2000-180810 A (published on Jun. 30, 2000)

[Patent Document 2] JP 2014-203365 A (published on Oct. 27, 2014)

[Patent Document 3] JP 2004-198124 A (published on Jul. 15, 2004)

[Patent Document 4] JPH 11-85235 A (published on Mar. 30, 1999)

[Patent Document 5] JP 2012-242315 A (published on Dec. 10, 2012)

[Patent Document 6] JP 2014-137815 A (published on Jul. 28, 2014)

To increase the accuracy of positioning or inspection, the resolution ofan imaging device has to be increased; that is, the resolution of theimaging device has to be enhanced. However, if the resolution of theimaging device is enhanced, the number of pixels is increased, and theimage transfer time and the image processing time are thereby increased.A specific description will be given with reference to FIG. 17. FIG. 17is a diagram illustrating a relationship between the number of pixelsand a processing time. As illustrated in FIG. 17, the processing time isincreased as the number of pixels is increased from 300,000 to 12million.

Accordingly, accuracy enhancement and speed enhancement have a trade-offrelationship, and it is difficult to achieve a good balance between thetwo.

Moreover, the techniques described in Patent Documents 1 and 2 above donot take any measures to achieve both enhancement of accuracy andreduction in the processing time.

SUMMARY

An aspect may realize an image processing device and the like forreducing the processing time while increasing the accuracy ofpositioning or inspection.

To solve the problems described above, an image processing deviceaccording to an aspect is an image processing device that is used forposition control of a target object and that is configured to detect aposition of the target object from a captured image captured by animaging device, the image processing device including a predicting unitconfigured to predict a current position of the target object, asearching unit configured to search for a position of the target objectfrom a predetermined region that is a partial region of the capturedimage and that includes a position predicted by the predicting unit, andan outputting unit configured to output the position of the targetobject retrieved by the searching unit.

According to the configuration described above, in the case ofcontrolling the position of a target object by position control, theposition of the target object may be measured by searching only apredetermined region, in a captured image captured by the imagingdevice, including a predicted current position of the target object.This allows the amount of processing to be reduced compared to a case ofsearching the entire captured image. Also, even if the resolution of theimaging device is enhanced, the amount of processing is notsignificantly increased, because search is performed only on a part of acaptured image. Accordingly, the processing time can be reduced whileincreasing the accuracies of positioning of the target object andinspection of the target object.

According to the image processing device according to an aspect, thepredicting unit may predict the current position of the target object byusing instruction information from a control device configured tocontrol a position of the target object.

The instruction information from the control device is for controllingthe target object, and when the information is used, the movement of thetarget object may be predicted. According to the configuration describedabove, the current position of the target object is predicted by usingthe instruction information from the control device, and thus, accurateprediction may be performed.

According to the image processing device according to an aspect, thepredicting unit may predict the current position of the target object byusing a change in a position of the target object in a plurality ofcaptured images captured by the imaging device.

If the change in the position of the target object up to the currentposition is monotonic, the change in the position up to then and thechange in the position from then on are highly likely the same.According to the configuration described above, the current position ofthe target object is predicted from the change in the position of thetarget object in a plurality of captured images, and thus, if the changein the position is monotonic, the current position may be appropriatelypredicted.

The image processing device according to an aspect may include anotifying unit configured to notify the imaging device of thepredetermined region, and an obtaining unit configured to obtain a partof the captured image corresponding to the predetermined region from theimaging device, where the searching unit may search for the position ofthe target object from the part, obtained by the obtaining unit, of thecaptured image corresponding to the predetermined region.

According to the configuration described above, only the captured imageof a predetermined region which is a search target is obtained from theimaging device, and thus, the amount of data of an image that istransmitted from the imaging device may be reduced. The processing timemay thereby be reduced.

The image processing device according to an aspect may include anobtaining unit configured to obtain the captured image from the imagingdevice, and a region extracting unit configured to extract thepredetermined region from the captured image obtained by the obtainingunit, where the searching unit may search for the position of the targetobject from the predetermined region extracted by the region extractingunit.

According to configuration described above, it is possible to extractonly the predetermined region, which is a search target, from thecaptured region. Accordingly, it is possible to search only thepredetermined region.

According to the image processing device according to an aspect, aplurality of the imaging devices may be present, the predicting unit maypredict the current position of the target object from a first capturedimage that is captured by a first imaging device that is one of theplurality of imaging devices, and the searching unit may search for theposition of the target object from a second captured image that iscaptured by a second imaging device that is another one of the pluralityof imaging devices, the second captured image capturing a predeterminedregion including a position predicted by the predicting unit.

According to the configuration described above, the first imaging devicemay perform capturing of the whole image, and capturing may be performedby the second imaging device by using only a part of the foregoingresult to thereby search for the position of the target object. A moreaccurate position of the target object may thereby be grasped.

To solve the problems described above, a control system according to oneor more aspects includes the image processing device and a conveyingdevice, where a position of the target object is moved by the conveyingdevice, and the first imaging device is installed on an upstream side ofthe conveying device than the second imaging device.

According to the configuration described above, the first imaging deviceis installed on the upstream side of the conveying device, and thesecond imaging device is installed on the downstream side than the firstimaging device, and thus, prediction of the position of the targetobject performed using a captured image of the first imaging device maybe appropriately reflected in an imaging position of the second imagingdevice. Accordingly, the second imaging device may accurately capture animage of the target object, and the position of the target object may beaccurately grasped. Also, the accuracy of inspection may be increased.

To solve the problems described above, a control system according to oneor more aspects includes the image processing device, a control deviceconfigured to control a position of the target object, and an imagingdevice configured to capture an image of a region including the targetobject.

According to the configuration described above, the same effect as theforegoing effect may be achieved.

To solve the problems described above, a control method of an imageprocessing device according to one or more aspects is a control methodof an image processing device that is used for position control of atarget object and that is configured to detect a position of the targetobject from a captured image captured by an imaging device, the controlmethod including predicting a current position of the target object,searching for a position of the target object from a predeterminedregion that is a partial region of the captured image and that includesa position predicted in the predicting, and outputting the position ofthe target object retrieved in the searching.

According to the configuration described above, the same effect as theforegoing effect may be achieved.

The image processing device according to each aspect on may be realizedby a computer, and in this case, a control program of an imageprocessing device which causes the computer to realize the imageprocessing device by causing the computer to operate as each unit(software element) provided to the image processing device, and acomputer-readable recording medium recording the control program arealso included within the scope of the present invention.

According to an aspect, an effect that the amount of processing can bereduced compared to a case where search is performed on the entirecaptured image can be achieved. Also, even if the resolution of theimaging device is enhanced, the amount of processing is notsignificantly increased, because search is performed only on a part ofthe captured image. Accordingly, an effect that the processing time canbe reduced while increasing the accuracies of positioning of the targetobject and inspection of the target object can be achieved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a main configuration of an imageprocessing device according to an embodiment;

FIG. 2 is a diagram schematically illustrating a control systemaccording to an embodiment;

FIGS. 3A and 3B are diagrams illustrating a flow of processing of animage processing device, where FIG. 3A is a flowchart illustrating aflow of processing, and FIG. 3B is a diagram illustrating processing ofa control system in a simplified manner;

FIG. 4 is a diagram illustrating an outline of a method for restrictinga search region;

FIGS. 5A to 5D are diagrams illustrating a method for obtaining apredetermined region image which is a target to be analyzed by anobtained image analyzer;

FIG. 6 is a diagram illustrating a comparison of a processing timeaccording to a conventional technique and a processing time according toan embodiment;

FIGS. 7A and 7B are diagrams illustrating examples of a predeterminedregion;

FIG. 8 is a diagram illustrating an effect of an embodiment;

FIG. 9 is a block diagram illustrating a main configuration of an imageprocessing device according to an embodiment;

FIGS. 10A and 10B are diagrams illustrating a flow of processing of aimage processing device according to an embodiment, where FIG. 10A is aflowchart illustrating a flow of processing, and FIG. 10B is a diagramillustrating processing of a control system in a simplified manner;

FIG. 11 is a diagram illustrating a method for calculating a velocityvector of a target object X;

FIG. 12 is a block diagram illustrating a main configuration of an imageprocessing device according to an embodiment;

FIGS. 13A to 13C are diagrams illustrating an example of a controlsystem according to an embodiment, where FIG. 13A is a diagramillustrating an outline of a control system, FIG. 13B is a diagramillustrating an example image that is captured by a first imaging unit,and FIG. 13C is a diagram illustrating an example image that is capturedby a second imaging unit;

FIG. 14 is a flowchart illustrating a flow of processing of an imageprocessing device according to an embodiment;

FIG. 15 is a diagram illustrating an outline of a control systemaccording to an embodiment;

FIG. 16 is a diagram illustrating predetermined regions according to anembodiment; and

FIG. 17 is a diagram illustrating a relationship between the number ofpixels and a processing time.

DETAILED DESCRIPTION First Embodiment

Hereinafter, an embodiment will be described in detail. First, anoutline of a control system 100 according to an embodiment will bedescribed with reference to FIG. 2. FIG. 2 is a diagram schematicallyillustrating the control system 100.

As illustrated in FIG. 2, briefly describing, the control system 100according to an embodiment includes a control device 1, a servo driver3, a control target 5 (servo motor 4), an image processing device 10,and an imaging unit (imaging device) 20. An instruction value generatedby the control device 1 is input to the servo driver 3, and the servodriver 3 drives the servo motor 4 by torque that is based on thereceived instruction value so as to perform position control of a targetobject X at the control target 5. In an embodiment, an XYθ stage isassumed as the control target 5, and the position of the target object Xon the XYθ stage is to be controlled.

Also, the imaging unit 20 captures an image of the control target 5, andthe captured image is processed by the image processing device 10. Thecontrol device 1 is thereby notified of the accurate position of thetarget object X at the control target 5, and accurate position controlis thus realized. By realizing accurate position control, accuratepositioning of the target object X at the control target 5, and accurateinspection at the control target 5 may be realized, for example.

Moreover, according to an embodiment, the amount of processing isreduced by predicting the position of the target object X at the controltarget 5 and thereby restricting a search region in a captured imagecaptured by the imaging unit 20. A high-resolution captured image maythereby be processed in a short time.

A specific description will be given with reference to FIG. 4. FIG. 4 isa diagram illustrating an outline of a method for restricting a searchregion. FIG. 4 illustrates an example of a captured image 401, which isvertically Cy and horizontally Cx and which captures the control target5, and the positions of the target object X over time are indicated by xmarks for the sake of convenience. FIG. 4 indicates the lapse of time by“t−2”, “t−1” and “t”, and the position of the target object X at thetime “t−2”, the position of the target object X at the time “t−1”, andthe position of the target object X at the time “t” are indicated. In anembodiment, to grasp the accurate position of the target object X at thetime “t”, the position of the target object X at the time “t” ispredicted by using the preceding time “t−1” (or depending on thesituation, the second preceding time “t−2” is also used), and search isperformed only in the part, in the captured image, of a predeterminedregion 402 including the predicted position. This eliminates the need tosearch the entire captured image 401, and even a high-resolutioncaptured image may be processed in a short time. For example, if arectangular region which is vertically My and horizontally Mx is used asa model, a rectangular region which is vertically My+d2 and horizontallyMx+d1 is conceivable as the predetermined region 402.

Configuration of Image Processing Device

Next, a main configuration of the image processing device 10 will bedescribed with reference to FIG. 1. FIG. 1 is a block diagramillustrating a main configuration of the image processing device 10.

As illustrated in FIG. 1, the image processing device 10 includes apredicted position calculator (predicting unit) 11, a capturing regiondesignating unit 12, an imaging controller (notifying unit) 13, acaptured image obtaining unit (obtaining unit) 14, an obtained imageanalyzer (searching unit, region extracting unit) 15, and an analysisresult outputting unit (outputting unit) 16.

The predicted position calculator 11 predicts a current position of thetarget object X at the control target 5. Then, the capturing regiondesignating unit 12 is notified of the predicted position. Morespecifically, the predicted position calculator 11 obtains driveinformation (region restriction information, instruction information)for driving the control target 5 from the control device 1, and predictsthe current position of the target object X from the obtained driveinformation and the previous position of the target object X.Additionally, the predicted position calculator 11 may predict theattitude of the target object X together with the position. This allowsa predetermined region, described below, to be set according to theattitude.

The capturing region designating unit 12 designates a predeterminedregion including the predicted current position of the target object Xnotified by the predicted position calculator 11, and notifies theimaging controller 13 of the predetermined region.

The imaging controller 13 notifies the imaging unit 20 of thepredetermined region notified by the capturing region designating unit12, and causes the imaging unit 20 to capture an image of the controltarget 5.

The captured image obtaining unit 14 obtains, from the imaging unit 20,the part corresponding to the predetermined region (predetermined regionimage) in the captured image captured by the imaging unit 20, andtransmits the image to the obtained image analyzer 15.

The obtained image analyzer 15 analyzes the predetermined region imagetransmitted from the captured image obtaining unit 14, searches for thetarget object X, and measures (detects) the position of the targetobject X. Then, the analysis result outputting unit 16 is notified ofthe measurement result. The method for measuring the position of thetarget object X by analyzing an image can be realized by a well-knowntechnique, and description thereof is omitted.

The analysis result outputting unit 16 outputs the measurement resultnotified by the obtained image analyzer 15 to an external device or thelike. Additionally, the image processing device 10 may include a displayunit or the like, and the display unit may be caused to output themeasurement result.

The imaging unit 20 captures an image of the control target 5 accordingto an instruction from the imaging controller 13. Then, of the capturedimage, only a predetermined region image corresponding to thepredetermined region instructed by the imaging controller 13 istransmitted to the image processing device 10.

Flow of Processing of Image Processing Device

Next, a flow of processing of the image processing device 10 will bedescribed with reference to FIGS. 3A and 3B. FIGS. 3A and 3B arediagrams illustrating a flow of processing of the image processingdevice 10, where FIG. 3A is a flowchart illustrating the flow ofprocessing, and FIG. 3B is a diagram illustrating processing of thecontrol system 100 in a simplified manner.

As illustrated in FIG. 3A, when control processing by the control system100 is started, the imaging controller 13 causes the imaging unit 20 tocapture an image of the control target 5 (S101). Next, the obtainedimage analyzer 15 measures the position of the target object X from thecaptured image (S102: workpiece search process). Next, the predictedposition calculator 11 predicts the position of the target object X atthe time of the next capturing, from the drive information regarding thecontrol target 5 (instruction value for the servo motor 4) obtained fromthe control device 1 and the current position of the target object X(S103, prediction step). Then, the capturing region designating unit 12designates, from the predicted position, a region including the positionas the predetermined region (S104).

Then, the imaging controller 13 transmits information indicating thepredetermined region to the imaging unit 20, and also, causes theimaging unit 20 to capture an image of the control target 5 (S105). Theimaging unit 20 transmits a predetermined region image, in the capturedimage, corresponding to the predetermined region to the image processingdevice 10, and the obtained image analyzer 15 analyzes the predeterminedregion image, and measures (searches for) the position of the targetobject X (S106, search step). Then, the analysis result outputting unit16 outputs the measurement result (S107, output step).

Another Example of Method for Obtaining Predetermined Region Image asAnalysis Target

Next, another example of the method for obtaining a predetermined regionimage which is a target to be analyzed by the obtained image analyzer 15will be described with reference to FIGS. 5A to 5D. FIGS. 5A to 5D arediagrams for describing a method for obtaining a predetermined regionimage which is a target to be analyzed by the obtained image analyzer15.

According to the configuration described above, an image that istransmitted from the imaging unit 20 to the image processing device 10is the predetermined region image. That is, with reference to a capturedimage 501 as illustrated in FIGS. 5A and 5B, only a predetermined regionimage 511 in the captured image 501 is transmitted to the imageprocessing device 10 in the example illustrated in FIG. 5A, and onlypredetermined region images 511, 512, 513, 514 in the captured image 501are transmitted to the image processing device 10 in the exampleillustrated in FIG. 5B. Additionally, FIG. 5B illustrates an example ofa case where there are a plurality of target objects X.

Such a configuration is not restrictive, and for example, the image thatis transmitted from the imaging unit 20 to the image processing device10 may be the captured image itself, that is, the image capturing theentire control target 5. In this case, the image processing device 10which has obtained the captured image may analyze, by the obtained imageanalyzer 15, only the region, in the captured image, designated by thecapturing region designating unit 12. That is, as illustrated in FIG.5C, the image processing device 10 obtains the captured image 501, andthe obtained image analyzer 15 analyzes only a predetermined regionimage 521, in the captured image 501, corresponding to the predeterminedregion. Or as illustrated in FIG. 5D, the image processing device 10obtains the captured image 501, and the obtained image analyzer 15analyzes only predetermined region images 521, 522, 523, 524, in thecaptured image 501, corresponding to the predetermined regions.

Also with such a configuration, the analysis target is not the entirecaptured image, but only a predetermined region in the captured image,and thus, the amount of processing may be reduced compared to a case ofanalyzing the entire captured image.

Processing Time

Next, to what extent the processing time is reduced by an embodimentwill be described with reference to FIG. 6. FIG. 6 is a diagramcomparing a processing time according to a conventional technique and aprocessing time according to an embodiment.

As illustrated in FIG. 6, according to a conventional technique, thetime taken for an imaging process (including the time of image transferfrom the imaging device to the image processing device) is 33 ms, animage processing time for analysis of a captured image is 30 ms, and aprocess for outputting an analysis result takes 1 ms. That is, accordingto the conventional technique, the entire processing time is 33+30+1=64ms.

By contrast, in an embodiment, firstly, a predetermined region image istransmitted from the imaging unit 20 to the image processing device 10,and thus, the transmission time is reduced, and the time taken for theimaging process is 10 ms. Also, the analysis target is only thepredetermined region image, and the processing time is reduced, and theimage processing time is 10 ms. The time taken for the output process isnot changed. As a result, the entire processing time according to anembodiment is 10+10+1=21 ms, and is reduced by about 66% compared to thecase of the conventional technique.

Furthermore, also in the case where the captured image to be transmittedfrom the imaging unit 20 is unchanged but the region as an analysistarget is restricted by the image processing device 10, the imageprocessing time is reduced to 10 ms. Accordingly, the entire processingtime in this case is 33+10+1=44 ms, and is reduced by about 31% comparedto the case of the conventional technique.

Example of Predetermined Region

Next, examples of the predetermined region designated by the capturingregion designating unit 12 will be described with reference to FIGS. 7Aand 7B. FIGS. 7A and 7B are diagrams illustrating examples of thepredetermined region. As described above, the predetermined region maybe a rectangular region including the target object X (regions 721, 722,723 in FIG. 7A), or may be a region according to the shape of the targetobject X as illustrated in FIG. 7B (regions 711, 712, 713, 714 in FIG.7B).

Effect of Present Invention

Next, an effect of an embodiment will be described with reference toFIG. 8. FIG. 8 is a diagram for describing an effect of an embodiment.As illustrated in FIG. 8, it can be seen that, according to theconventional technique, the processing time is significantly increasedas the number of pixels of the imaging device is increased.

On the other hand, according to an embodiment, the processing time isnot greatly increased even when the number of pixels is increased.Additionally, in FIG. 8, an embodiment A is a case where processing isperformed in a standard mode, and an embodiment B is a case whereprocessing is performed in a high-speed alignment mode.

The standard mode corresponds to the configuration described above wherethe imaging unit 20 transmits the entire captured image to the imageprocessing device 10 and where the image processing device 10 restrictsthe search region, and the high-speed alignment mode corresponds to theconfiguration described above where the image transmitted from theimaging unit 20 to the image processing device 10 is a predeterminedregion image.

Second Embodiment

Another embodiment will be described below with reference to FIGS. 9 to11. Additionally, for the sake of convenience, members having the samefunction as the members described in the embodiment above will bedenoted with the same reference signs, and description thereof will beomitted.

An embodiment is different from the first embodiment described abovewith respect to the method for predicting the current position of atarget object X. First, a main configuration of an image processingdevice 10 a according to an embodiment will be described with referenceto FIG. 9. FIG. 9 is a block diagram illustrating a main configurationof the image processing device 10 a.

As illustrated in FIG. 9, the image processing device 10 a according toan embodiment includes a predicted position calculator (predicting unit)11 a instead of the predicted position calculator 11 of the imageprocessing device 10 according to the first embodiment described above,and includes a captured image obtaining unit (obtaining unit) 14 ainstead of the captured image obtaining unit 14.

The predicted position calculator 11 a predicts the current position ofa target object X from a change in the position of the target object Xin a plurality of captured images obtained by the captured imageobtaining unit 14 a. Details of the prediction method will be givenbelow.

The captured image obtaining unit 14 a transmits a captured imageobtained from the imaging unit 20 to the obtained image analyzer 15, andalso to the predicted position calculator 11 a.

Next, a flow of processing of the image processing device 10 a will bedescribed with reference to FIGS. 10A and 10B. FIGS. 10A and 10B arediagrams illustrating a flow of processing of the image processingdevice 10 a, where FIG. 10A is a flowchart illustrating the flow ofprocessing, and FIG. 10B is a diagram illustrating processing of thecontrol system 100 in a simplified manner.

As illustrated in FIG. 10A, when control processing by the controlsystem 100 is started, the imaging controller 13 causes the imaging unit20 to capture an image of the control target 5 (S201). Next, theobtained image analyzer 15 measures the position of the target object Xfrom the captured image (S202: workpiece search process). Then, theimaging controller 13 causes the imaging unit 20 to capture an image ofthe control target 5 in the same manner (S203). Also, the obtained imageanalyzer 15 measures the position of the target object X from thecaptured image (S204: workpiece search process).

Next, the predicted position calculator 11 a calculates a velocityvector of the target object X by using the captured images obtained fromthe captured image obtaining unit 14 a, and predicts the position of thetarget object X at the time of the next capturing (S205). Then, thecapturing region designating unit 12 designates, from the predictedposition, a region including the position as the predetermined region(S206).

Then, the imaging controller 13 transmits information indicating thepredetermined region to the imaging unit 20, and also, causes theimaging unit 20 to capture an image of the control target 5 (S207). Theimaging unit 20 transmits a predetermined region image, in the capturedimage, corresponding to the predetermined region to the image processingdevice 10, and the obtained image analyzer 15 analyzes the predeterminedregion image, and measures the position of the target object X (S208).Then, the analysis result outputting unit 16 outputs the measurementresult (S209).

Next, a method for calculating the velocity vector of the target objectX will be described with reference to FIG. 11. FIG. 11 is a diagram fordescribing a method for calculating the velocity vector of the targetobject X.

In FIG. 11, P indicates the position of the target object X, Vxindicates the velocity vector in the x-direction, and Vy indicates thevelocity vector in the y-direction. Also, Pra indicates the upper leftposition of a predetermined region, and Prb indicates the lower rightposition of the predetermined region.

In FIG. 11, P_(t−2)=(X_(t−2), y_(t-2)) indicates the position of thetarget object X at the time of second previous capturing, andP_(t−1)=(x_(t−1), y_(t−1)) indicates the position of the target object Xat the time of previous capturing.

First, the predicted position calculator 11 a determines the velocityvector at the time of the target object X moving from P_(t−2) toP_(t−1). Here, the x-direction component of the velocity vector isVx_(t−1), and the y-direction component is Vy_(t−1).

The predicted position calculator 11 a predicts a current position P_(t)of the target object X by using the following equation.

P _(t)=(x _(t) , y _(t))=(x _(t−1) +Vx _(t−1) ×Δt, y _(t−1) +Vy _(t−1)×Δt)

Here, Δt is a capturing time interval.

Also, the capturing region designating unit 12 sets an upper leftposition Pra_(t) of the predetermined region to (x_(t)−Δd, y_(t)−Δd),and sets a lower right position Prb_(t) of the predetermined region to(x_(t)+Δd, y_(t)+Δd). The size of the predetermined region is indicatedby Δd, and Δd may be arbitrarily set. Additionally, in this case, thex-direction and the y-direction take the same value (that is, thepredetermined region is a square), but this is not restrictive, and thepredetermined region may alternatively be rectangular with thex-direction being Δd₁ and the y-direction being Δd₂.

Third Embodiment

Further another embodiment will be described below with reference toFIGS. 12 to 16. Additionally, for the sake of convenience, membershaving the same function as the members described in the embodimentsabove will be denoted with the same reference signs, and descriptionthereof will be omitted.

An embodiment assumes a control system according to which the controltarget 5 is a not an XYθ stage but a conveying device (such as aconveyor belt), and according to which a target object X moving on thecontrol target 5 is picked up at a predetermined position.

First, a main configuration of an image processing device 10 b accordingto an embodiment will be described with reference to FIG. 12. FIG. 12 isa block diagram illustrating a main configuration of the imageprocessing device 10 b. As illustrated in FIG. 12, the image processingdevice 10 b according to an embodiment includes the predicted positioncalculator 11 a, the capturing region designating unit 12, the imagingcontroller (notifying unit) 13 a, an imaging controller 13 b, a capturedimage obtaining unit (obtaining unit) 14 b, a captured image obtainingunit 14 c, the obtained image analyzer 15, and the analysis resultoutputting unit 16.

Furthermore, the control system 100 includes a first imaging unit(imaging device, first imaging device) 20A and a second imaging unit(imaging device, second imaging device) 20B, instead of the imaging unit20 according to the embodiments described above.

The imaging controller 13 a causes the first imaging unit 20A to capturean image of the control target 5. The imaging controller 13 b causes thesecond imaging unit 20B to capture an image of the control target 5. Thecaptured image obtaining unit 14 b obtains a captured image from thefirst imaging unit 20A. The captured image obtaining unit 14 c obtains acaptured image from the second imaging unit 20B.

The first imaging unit 20A is arranged on the upstream side of thesecond imaging unit 20B. Additionally, the second imaging unit 20B mayhave a higher resolution than the first imaging unit 20A.

A more detailed description will be given with reference to FIGS. 13A to13C. FIGS. 13A to 13C are diagrams illustrating an example of thecontrol system according to an embodiment, where FIG. 13A is a diagramillustrating an outline of the control system, FIG. 13B is a diagramillustrating an example image that is captured by the first imaging unit20A, and FIG. 13C is a diagram illustrating an example image that iscaptured by the second imaging unit 20B.

As illustrated in FIG. 13A, according to an embodiment, a target objectX is moved by the control target 5, and is eventually picked up by arobot 50. The second imaging unit 20B is arranged in a manner capable ofcapturing the position of the control target 5 corresponding to thepick-up position of the robot 50, and the first imaging unit 20A isarranged (installed) on the upstream side of the second imaging unit20B.

As illustrated in FIG. 13B, an image of the control target 5 is capturedby the first imaging unit 20A on the upstream side and the position ofthe target object X is measured from the captured image, and the secondimaging unit 20B on the downstream side captures an image of a position,predicted from the measured position, corresponding to the position ofthe target object X. In the example illustrated in FIG. 13C, capturingregions 1311, 1312, 1313, 1314 are the capturing regions of the secondimaging unit 20B.

This allows the position of the target object X to be more accuratelymeasured (grasped), and a pick-up process of the target object X by therobot 50 may be accurately performed.

Next, a flow of processing of the image processing device 10 b will bedescribed with reference to FIGS. 14 and 15. FIG. 14 is a flowchartillustrating a flow of processing of the image processing device 10 b.Also, FIG. 15 is a diagram illustrating an outline of the control system100 according to an embodiment.

As illustrated in FIG. 14, when control processing by the control system100 is started, the imaging controller 13 a causes the first imagingunit 20A to capture an image of the control target 5 (S301). Next, thepredicted position calculator 11 a measures the position of the targetobject X from the captured image captured by the first imaging unit 20A(S302: workpiece search process). Then, the predicted positioncalculator 11 a assigns an index to the target object X (S303).Specifically, as illustrated in FIG. 15, an index such as i=0, . . . , 3is assigned to a target object X present in a capturing region (firstregion) that is captured by the first imaging unit 20A. Then, theimaging controller 13 a causes the first imaging unit 20A to capture animage of the control target 5 in the same manner (S304). Also, thepredicted position calculator 11 a measures the position of the targetobject X from the captured image (first captured image) captured by thefirst imaging unit 20A (S305: workpiece search process). Then, thepredicted position calculator 11 a performs association of indices withrespect to the target object X (S306). Specifically, the same index isassigned to a target object X that is the same target object X to whichan index was previously assigned. If there is a target object X which isnot assigned with an index (S307: YES), an index is assigned to thistarget object X (S308).

Then, the predicted position calculator 11 a calculates the velocityvector of the target object X by using the two captured images, andpredicts the movement position of the target object X (S309).

Then, step S304 is performed again, and also, the process in step S310is performed.

In step S310, the capturing region designating unit 12 designates apredetermined region including the position predicted in step S309(S310).

Then, the imaging controller 13 b transmits information indicating thepredetermined region to the second imaging unit 20B, and also, causesthe second imaging unit 20B to capture an image of a region, of thecontrol target 5, corresponding to the predetermined region (forexample, a second region in FIG. 15) (S311). Then, the second imagingunit 20B transmits the captured image to the image processing device 10b, and the obtained image analyzer 15 analyzes the captured image(second captured image), and measures the position of the target objectX (S312). Then, the analysis result outputting unit 16 outputs themeasurement result (S313).

Next, predetermined regions will be described with reference to FIG. 16.FIG. 16 is a diagram for describing predetermined regions according toan embodiment. A capturing region 1601 in FIG. 16 is a regioncorresponding to the capturing region of the first imaging unit 20A. Thecapturing region designating unit 12 designates, as predeterminedregions, regions 1611, 1612, 1613, which are regions including positionspredicted from positions of target objects X in a captured imagecaptured by the first imaging unit 20A.

Example Implementation by Software

A control block of the image processing device 10 (10 a, 10 b)(particularly, the predicted position calculator 11 (11 a), thecapturing region designating unit 12, the imaging controller 13 (13 a,13 b), the captured image obtaining unit 14 (14 a, 14 b, 14 c), theobtained image analyzer 15, the analysis result outputting unit 16) maybe realized by a logic circuit (hardware) formed to an integratedcircuit (IC chip) or the like, or may be realized by software using acentral processing unit (CPU).

In the latter case, the image processing device 10 (10 a, 10 b) includesa CPU that executes instructions of a program that is software realizingeach function, a read only memory (ROM) or a storage device (eachreferred to as “recording medium”) storing the program and variouspieces of data in such a form that they are readable by a computer (or aCPU), and a random access memory (RAM) that develops the program, forexample. An aspect can be achieved by a computer (or a CPU) reading andexecuting the program stored in the recording medium. As the storagemedium, “a non-transitory tangible medium” such as a tape, a disk, acard, a semiconductor memory, or a programmable logic circuit may beused. Also, the program may be provided to the computer via anytransmission medium (such as a communication network or a broadcastwave) which is capable of transmitting the program. Additionally, anaspect can also be implemented by the program which is embodied byelectronic transmission as a data signal embedded in a carrier wave.

The present invention is not limited to the embodiments described above,and various modifications may be made within the scope of the claims,and embodiments combining technical means disclosed in differentembodiments are also included in the technical scope of the presentinvention.

1. An image processing device that is used for position control of atarget object and that is configured to detect a position of the targetobject from a captured image captured by an imaging device, the imageprocessing device comprising: a predicting unit configured to predict acurrent position of the target object; a searching unit configured tosearch for a position of the target object from a predetermined regionthat is a partial region of the captured image and that includes aposition predicted by the predicting unit; and an outputting unitconfigured to output the position of the target object retrieved by thesearching unit.
 2. The image processing device according to claim 1,wherein the predicting unit predicts the current position of the targetobject by using instruction information from a control device configuredto control a position of the target object.
 3. The image processingdevice according to claim 1, wherein the predicting unit predicts thecurrent position of the target object by using a change in a position ofthe target object in a plurality of captured images captured by theimaging device.
 4. The image processing device according to claim 1,comprising: a notifying unit configured to notify the imaging device ofthe predetermined region; and an obtaining unit configured to obtain apart of the captured image corresponding to the predetermined regionfrom the imaging device, wherein the searching unit searches for theposition of the target object from the part, obtained by the obtainingunit, of the captured image corresponding to the predetermined region.5. The image processing device according to claim 1, comprising: anobtaining unit configured to obtain the captured image from the imagingdevice; and a region extracting unit configured to extract thepredetermined region from the captured image obtained by the obtainingunit, wherein the searching unit searches for the position of the targetobject from the predetermined region extracted by the region extractingunit.
 6. The image processing device according to claim 1, wherein aplurality of the imaging devices are present, wherein the predictingunit predicts the current position of the target object from a firstcaptured image that is captured by a first imaging device that is one ofthe plurality of imaging devices, and wherein the searching unitsearches for the position of the target object from a second capturedimage that is captured by a second imaging device that is another one ofthe plurality of imaging devices, the second captured image capturing apredetermined region including a position predicted by the predictingunit.
 7. A control system comprising: the image processing deviceaccording to claim 6; and a conveying device, wherein a position of thetarget object is moved by the conveying device, and wherein the firstimaging device is installed on an upstream side of the conveying devicethan the second imaging device.
 8. A control system comprising: theimage processing device according to claim 1; a control deviceconfigured to control a position of the target object; and an imagingdevice configured to capture an image of a region including the targetobject.
 9. A control method of an image processing device that is usedfor position control of a target object and that is configured to detecta position of the target object from a captured image captured by animaging device, the control method comprising: predicting a currentposition of the target object; searching for a position of the targetobject from a predetermined region that is a partial region of thecaptured image and that includes a position predicted in the predicting;and outputting the position of the target object retrieved in thesearching.
 10. A non-transitory computer-readable recording mediumstoring a control program for causing a computer to function as theimage processing device according to claim 1, wherein the computer iscaused to function as the predicting unit, the searching unit, and theoutputting unit.
 11. The image processing device according to claim 2,comprising: a notifying unit configured to notify the imaging device ofthe predetermined region; and an obtaining unit configured to obtain apart of the captured image corresponding to the predetermined regionfrom the imaging device, wherein the searching unit searches for theposition of the target object from the part, obtained by the obtainingunit, of the captured image corresponding to the predetermined region.12. The image processing device according to claim 3, comprising: anotifying unit configured to notify the imaging device of thepredetermined region; and an obtaining unit configured to obtain a partof the captured image corresponding to the predetermined region from theimaging device, wherein the searching unit searches for the position ofthe target object from the part, obtained by the obtaining unit, of thecaptured image corresponding to the predetermined region.
 13. The imageprocessing device according to claim 2, comprising: an obtaining unitconfigured to obtain the captured image from the imaging device; and aregion extracting unit configured to extract the predetermined regionfrom the captured image obtained by the obtaining unit, wherein thesearching unit searches for the position of the target object from thepredetermined region extracted by the region extracting unit.
 14. Theimage processing device according to claim 3, comprising: an obtainingunit configured to obtain the captured image from the imaging device;and a region extracting unit configured to extract the predeterminedregion from the captured image obtained by the obtaining unit, whereinthe searching unit searches for the position of the target object fromthe predetermined region extracted by the region extracting unit.